Skip to content

fix boundary and performance issues in MathUtil conversions - #94

Merged
pjfanning merged 1 commit into
apache:trunkfrom
pjfanning:mathutil-conversion-fixes
Aug 25, 2026
Merged

fix boundary and performance issues in MathUtil conversions#94
pjfanning merged 1 commit into
apache:trunkfrom
pjfanning:mathutil-conversion-fixes

Conversation

@pjfanning

Copy link
Copy Markdown
Member

Three issues in MathUtil, each covered by a new test.

safeFloatToInt accepted 2147483648f. Widening Integer.MAX_VALUE to float rounds it up to 2^31, so f > Integer.MAX_VALUE was false for exactly 2^31 and the (int) cast silently saturated to 2147483647 — the wrong answer from a method whose job is to reject out-of-range input. Comparing as double fixes it. The lower bound was already correct, since -2^31 is exactly representable, and safeDoubleToInt was never affected.

toBigInteger guarded a large negative scale but not a large positive one. A value like 1E-10000000 has integerDigits == -9999999 and scale == +10000000, so neither existing condition tripped and BigDecimal.toBigInteger() was reached. That calls setScale(0), which computes 10^scale: measured at just over 4 seconds to return 0 from a 12-character input, and at larger exponents it throws ArithmeticException: BigInteger would overflow supported range instead of the documented IllegalArgumentException. The string passes parseAsBigDecimal's length limit easily, and the path is reachable from parsed content via JavaDecimalHolder / XmlObjectBase.getBigIntegerValue(). Any such value truncates to zero, so return BigInteger.ZERO directly — which also makes every sub-1 conversion cheaper.

toBigInteger had no explicit null check, unlike every other method here. It already threw NullPointerException from stripTrailingZeros(), so this is consistency rather than a behaviour change; toLong and toInt inherit it.

Also added a comment on the existing scale() < -DEFAULT_MAX_NUMBER_CHARS condition, which reads as redundant but is not: for 1E+2147483647 the precision() - scale() subtraction overflows to Integer.MIN_VALUE and only the scale check catches it. And corrected a few copy-pasted javadoc @return types (parseAsDouble documented float).

Ran the impl.util and impl.values tests locally — all pass. Leaving the full suite to CI.

🤖 Generated with Claude Code

safeFloatToInt accepted 2147483648f: widening Integer.MAX_VALUE to float
rounds it up to 2^31, so the upper-bound check never tripped and the cast
silently saturated to 2147483647. Compare as double instead.

toBigInteger guarded a large negative scale but not a large positive one,
so a tiny value such as 1E-10000000 reached BigDecimal.toBigInteger(),
which computes 10^scale - several seconds of CPU to produce zero, and an
undocumented ArithmeticException at larger exponents. Such values truncate
to zero, so return that directly.

Also add an explicit null check to toBigInteger, matching the other
methods, and correct a few copy-pasted javadoc return types.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@pjfanning
pjfanning merged commit 050e30c into apache:trunk Aug 25, 2026
3 checks passed
@pjfanning
pjfanning deleted the mathutil-conversion-fixes branch August 25, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant